This invention relates to the analysis of temporally related patterns by neural-like networks, and more particularly to pattern recognition by simulated adaptation of neural-like synapses based on temporal stimuli.
Prior neural-like networks have encoded temporal data for input to static networks, or employed back-propagation through time, avalanche filters, recursive networks and temporal difference learning.
In prior attempts to learn temporal relationships, use has been made of varying amplification weights, coupled with a constant periodic sampling of input signals. For temporal pattern recognition, the most common approach has been to adapt neural-like networks to handle the temporal aspects of problems by the use of sampled time windows, delayed feedback loops and other common approaches for dealing with temporal relationships. All of these approaches enable an engineer to use common weighting and adaptive techniques for the analysis of temporal events or signals.
It is an object of this invention is to avoid current neural network techniques and do better than to simply implement developed algorithms. A related object is to avoid the use of encoded temporal data for input to static networks, or the employment of back-propagation through time, avalanche filters, recursive networks and temporal difference learning.
Still another object of the invention is to avoid use of varying amplification weights, coupled with a constant periodic sampling of input signals.
A further object of the invention for temporal pattern recognition is to avoid adapting neural-like networks to handle the temporal aspects by sampled time windows, delayed feedback loops and other related approaches. A related object is to avoid use of common weighting and adaptive techniques for the analysis of temporal events or signals.
Although the input-output response of the common perceptron, i.e., artificial neuron, is commonly modeled after biological neural activity, its behavior leaves behind any temporal aspects of signals between neurons, which transmit and receive pulses. The nonlinear function of the artificial neuron, whether it has a sigmoid or some other type of function, is selected to model the reaction of a biological neuron to the frequency of pulse occurrences. It may be considered to be a frequency transfer function from an input frequency to an output frequency. The input and output of such a neuron, because of the selection of pulse occurrence functionality, becomes a numerical signal, rather than a temporal signal.
The simple perceptron is not adapted to analyze temporal relationships between pulses. It uses frequency as an input in order to analyze signal magnitudes. For temporal processing by a network of perceptrons, other means of encoding for the effects of time must be applied.
Accordingly, another object of the invention is to provide perceptron input-output that takes into account the temporal aspects of signals between neurons, which transmit and receive pulses. A related object is to avoid modeling the reaction of a biological neuron to the frequency of pulse occurrences. Still another related object is to avoid a frequency transfer function from an input frequency to an output frequency. Another related object is to avoid the selection of pulse occurrence functionality, as a numerical signal, rather than a temporal signal.
Still another object is to avoid the simple perception which uses frequency as an input in order to analyze signal magnitudes and is not adapted to analyze temporal relationships between pulses.
Yet another object of the invention is to achieve temporal processing by a network of perceptrons which can encode for the effects of time.
Further background is provided by Durbin, Miall & Mitchison in The Computing Neuron, Addison-Wesley Publishers Ltd. 1989; Amit in Modeling Brain Function, Cambridge University Press. 1989; and Jumper, "Cellular Neural Structures: A Theory In Distributed Neural Learning Within A Neural Network", Proceedings IEEE Dual-Use Technologies & applications Conference, 1993, SUNY Instute of Technology at Utica/Rome.